GPT models via an on-demand local translation proxy - #7
Merged
Conversation
Claude Code speaks the Anthropic Messages API; the gateway serves its 8 GPT models only on the OpenAI-compat surface. A loopback CLIProxyAPI (port 8320) translates between the two. - Installer: installs CLIProxyAPI (existing binary > Homebrew > release tarball / windows zip), fetches the live gpt-* list from /models, and writes proxy-config.yaml (mode 600 — it holds the API key) plus a local bearer in proxy.key. Non-fatal throughout: Claude models never depend on it. - Launcher: `--model gpt-*` (or a gpt default) starts the proxy on demand and points Claude Code at it; Claude models keep the direct path. An explicit --model wins over the configured default. `--stop` kills the proxy; --doctor reports its state. - Windows: qbraid-proxy.ps1 (ensure|status|stop) owns the lifecycle; the .cmd detects the requested model and delegates. Verified live on a fresh delete+reinstall: claude-sonnet direct, gpt-5.6-sol auto-starting the proxy, gpt-5.4-mini reusing it, stop, doctor. GPT models cap tools at 128, so heavy MCP setups need --strict-mcp-config (README'd). tests/model-routing.sh pins requested_model() against the real definition; wired into CI with the others.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gateway serves 8 GPT models (
gpt-5.6-sol,gpt-5.4-mini, …) that ClaudeCode cannot reach: it speaks Anthropic Messages, and they live only on the
OpenAI-compat surface. This closes that gap without touching the gateway:
How
A loopback CLIProxyAPI on port 8320 translates Anthropic Messages ↔
chat-completions — the pattern proven in a spike earlier today, including
agentic tool use. The installer sets it up (existing binary → Homebrew →
release download), generates its config from the gateway's live model list,
and the launcher starts it only when a
gpt-*model is actually requested.Claude models keep the direct path and never depend on the proxy. Everything is
non-fatal: if proxy setup fails, Claude models still work and the error says
how to fix it.
Verified live (fresh delete + reinstall)
proxy configured for 8 GPT models (starts on demand)qbraid-code -p(claude, direct)--model gpt-5.6-sol(proxy auto-start)--model gpt-5.4-mini(proxy reuse)--doctor/--stoptests/model-routing.sh(6 cases) pins the model-selection logic against thereal function definition; all suites green, shellcheck clean, bash 3.2 parse OK.
Caveats stated in the README
toolsat 128 — heavy MCP setups need--strict-mcp-config./modelpicker is Claude Code's own UI and lists Claudemodels only; GPT is chosen at launch with
--model.qbraid-proxy.ps1, .cmd changes) are reviewed + CI-parsedbut not executed — same standing gap as the rest of the Windows half.
qbraid-external-site#107's companion allowlist needsqbraid-proxy.ps1added (done on that branch).